Remote control of medical devices using a virtual device interface

ABSTRACT

An interface system and method for controlling a magnetic surgery system by displaying a virtual image of the device, adjusting the configuration of the device or the actuation controls to be applied to the device until the configuration of the displayed device assumes the configuration desired by the user, or selecting a desired target location for the tip, and causing a set of actuation controls to be applied to the actual device to cause the actual device to assume the configuration of the virtual device or to steer the device tip to the desired location.

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority of prior provisional applicationSerial No. 60/401,670, filed Aug. 6, 2002, entitled Method and Apparatusfor Improved Magnetic Surgery Employing Virtual Device Interface, thedisclosure of which is incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

[0002] This invention relates to the remote control of medical devicesin a subject's body, and in particular to a user interface for operatinga remotely controllable medical device which employs a “virtual device”interface.

[0003] Advances in technology have resulted in systems that allow aphysician or other medical professional to remotely control theorientation of the distal end of a medical device. It is now fairlyroutine to steer the distal end of a medical device inside a subject'sbody by mechanically manipulating controls on the proximal end of themedical device. Recently magnetic navigation systems have been developedthat allow a physician to orient the distal end of a medical deviceusing the field of an external source magnet. Other systems have beendeveloped for the automated remote orientation of the distal end of amedical device, for example by operating magnetostrictive orelectrostrictive elements incorporated into the medical device. Howeverthe medical device is controlled, it can still be difficult for aphysician to visualize the procedure site (which is out of view insidethe patient's body), to selected the desired direction in which toorient the distal end of the medical device and communicate the selecteddirection to the system in order to orient the distal end of the medicaldevice in the selected direction.

[0004] As stated above, magnetic navigation systems have been developedwhich apply a controlled magnetic field to an operating region in asubject, to orient a magnetically responsive element on a medical devicein the operating region. Examples of such systems include Ritter et al.,U.S. Pat. No. 6,241,671, issued Jun. 5, 2001, for Open Field System ForMagnetic Surgery (incorporated herein by reference). Magnetic navigationsystems permit faster and easier navigation, and allow the devices to bemade thinner and more flexible than conventional mechanically navigateddevices which must contain pull wires and other components for steeringthe device. Because of the advances made in magnetic surgery systems andmagnetically responsive medical devices, the determination of theappropriate field direction, and instructing the magnetic surgery systemto apply the determined magnetic field are probably the most difficulttasks remaining in magnetically assisted medical procedures. Significantefforts have been made to help the user to visualize the procedure, andimprove the user's ability to control the magnetic surgery system duringthe procedure. There is often a lag between the direction of the appliedfield, and the actual direction of the distal end of the medical device.In some current systems, the user specifies a field direction, andmentally must take into account the lag between the applied field andthe actual device direction.

SUMMARY OF THE INVENTION

[0005] This invention provides a method and apparatus for controlling aflexible medical device in a subject's body which employs a virtualdevice interface, i.e. an interface using a physical or computationalmodel of the actual device, possibly including a computerized controlinterface for real-time/interactive navigational control of the device.Such a computerized device control interface would accept user inputfrom an input device (for example, a joystick) and interpret theseinputs through a computer to apply appropriate controls to drive thedevice tip according to a pre-defined mapping. Generally the method ofthis invention comprises displaying an image of the distal end portionof a “virtual” medical device, and allowing the user to manipulate thedisplayed image of the distal end portion of the virtual medical deviceinto a desired configuration (shape and/or orientation), and thenremotely operate the device to cause the distal end portion of theactual medical device to assume the desired configuration represented bythe image of the virtual device.

[0006] In a preferred embodiment of the system and method of thisinvention, the configuration of the displayed virtual medical device canbe controlled by identifying a target point to which the virtual medicaldevice configures itself to. In another preferred embodiment, controlsthat change the shape of virtual medical device, e.g., a deflectioncontrol and a rotation control are used to control configuration. Instill another preferred embodiment of the system and method of thisinvention, the configuration of the displayed virtual medical device canbe changed by changing at least one control parameter (e.g., the appliedmagnetic field for a magnetically controlled device) and updating theimage of the virtual medical device to show the configuration of thedistal end of the virtual medical device “as if” the at least onecontrol parameter was changed (in the particular case of a magneticallycontrolled device, as if the new field were applied). In anotherpreferred embodiment, the user can identify a target location and theinterface uses the virtual device model to determine the controlparameter(s) to cause the actual device to reach the target. In stillanother alternate preferred embodiment of the system and method of thisinvention, a surface of points of the possible positions of the distalend of the medical device is displayed. The user can then select a pointon the surface, and operate the system to automatically apply thecorrect magnetic field to cause the distal end of the medical device toalign in the direction of the selected point.

[0007] A preferred embodiment of this invention provides an interfacesystem and method for facilitating the specification and application ofa magnetic field to the operating region in a subject to control thedistal end of a medical device in the operating region. This inventioncan provide a method and apparatus for controlling a magnetic navigationsystem, and in particular provides an interface system and method forfacilitating the specification and application of a magnetic field tothe operating region in a patient to control the distal end of amagnetically enabled medical device in the operating region. However,the virtual device interface of the present invention is not so limited,and can be used with any system for controlling the configuration of amedical device and actuated by any of a variety of means. Furthermore,the interface can be used with any elongate medical device, includingguide wires, catheters, and endoscopes, etc.

[0008] The system and method of this invention allows the user tovisualize the configuration of the distal end portion of the medicaldevice before actually applying the control variable(s) used to modifythe configuration of the device. This is faster and easier to learn andto operate than many current user interfaces; for instance some presentmagnetic navigation interfaces require the user to specify a magneticfield direction without reference to the current orientation of thedistal end portion of the medical device. This also facilitatesautomating the procedure combining directional control with an advancerfor advancing the medical device. Furthermore the present invention letsthe user directly manipulate the distal end of the device in aninteractive manner through the use of an input device whose manipulationby the user is mapped by a computer to changes in actuation controlvariables which then produce a corresponding change in deviceconfiguration. The change in device configuration may be visuallydisplayed to the user through the use of any of a variety of imagingsystems so that the user has interactive control over the device. Theinput device for this purpose may for example be a joystick, graphicalmenu buttons, keyboard buttons, or any other choice that is known tothose skilled in the art. The mapping of input device manipulations tochanges in device actuation controls may be chosen in a variety of waysthat provide intuitive spatial control of the device. A selection ofpossible mapping choices can be offered to the user so that the user canpick the one that she or he deems most appropriate in a givennavigational circumstance. These and other features and advantages willbe in part apparent, and in part pointed out hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

[0009]FIG. 1 is a schematic view of magnetic surgery systemincorporating the interface system of the present invention;

[0010]FIG. 2 is a schematic view of a possible display screen forimplementing the interface system for a magnetic surgery systemconstructed according to the principles of a preferred embodiment ofthis invention;

[0011]FIG. 3A is an enlarged view of one of the bi-plane imagingdisplays from the interface shown in FIG. 2, showing a surface ofpossible positions for the distal end portion of the medical device;

[0012]FIG. 3B is an enlarged view of one of the bi-plane imagingdisplays from the interface shown in FIG. 2, showing surfaces ofpossible positions for the distal end portion of the medical device fordifferent free lengths l;

[0013]FIG. 4A is a schematic view of the bi-plane imaging displays fromthe interface shown in FIG. 2, illustrating the identification of apoint.

[0014]FIG. 4B is a schematic view of the bi-plane imaging displays fromthe interface shown in FIG. 2, illustrating the identification of apoint.

[0015]FIG. 4C is a schematic view of the bi-plane imaging displays fromthe interface shown in FIG. 2, illustrating the identification of apoint.

[0016]FIG. 5 is an enlarged view of the control panel for the interfaceshown in FIG. 2;

[0017]FIG. 6 is a graph showing the points that the fixed length distalend portion of a medical device can contact in a plane;

[0018]FIG. 7 is a representation of the distal end portion of themedical device, showing points x₁, x₂, x₃, and x₄;

[0019]FIG. 8 is a representation of the relation between the directionvectors and the field vector;

[0020]FIG. 9 is a representation of the relation between the directionvectors and {right arrow over (w)}, the unit vector orthogonal to {rightarrow over (v)} and {right arrow over (n)}₁;

[0021]FIG. 10 is representation of a display for an alternate displayfor a second embodiment of a user interface, that selectively employs avirtual medical device display;

[0022]FIG. 11 is a representation of an alternate display for a secondembodiment of a user interface that selectively employs a virtualmedical device display; and

[0023]FIG. 12 is a representation of a display from a third embodimentof a user interface, with screen buttons:

[0024]FIG. 13 is a representation of the LAO and RAO images from thethird embodiment of the user interface;

[0025]FIG. 14 is a representation of the RAO image of the thirdembodiment of the user interface in target mode, showing the targetcursor, the target, and the virtual device;

[0026]FIG. 15 is representation of the LAO and RAO images from the thirdembodiment of the user interface in target mode, showing the virtualdevice extending to a selected target;

[0027]FIG. 16 is a representation of the LAO image of third embodimentof the user interface, showing the virtual device after the change incontrol has been effected;

[0028]FIG. 17 is a representation of the LAO image of the thirdembodiment of the user interface, showing a further change in thevirtual device after a prior change in control has been affected; and

[0029]FIG. 18 is a side elevation view of a medical device constructedaccording to the principles of this invention, and adapted for controlby various embodiments of this invention.

[0030] Corresponding reference numerals indicate corresponding partsthroughout the several views of the drawings.

DETAILED DESCRIPTION OF THE INVENTION

[0031] The virtual device interface of the present invention can be usedwith any type of remotely controllable medical device, including forexample mechanically, electrically, and magnetically actuatable medicaldevices. One possible use of the invention is in the control ofmagnetically actuatable devices, such as with the magnetic navigationsystem shown in FIG. 1. While described below primarily in connectionwith a preferred embodiment in the form of a magnetic surgery system inconjunction with an X-ray imaging system, this invention is not solimited. For example, a Magnetic Resonance imaging system could be used,with the catheter being steered by torques generated by the use ofchangeable magnetic moments located on the device that interact with thestatic magnetic field of the Magnetic Resonance imaging system.Likewise, any other imaging modality or actuation modality could beemployed.

[0032] As shown in FIG. 1, a magnetic surgery system is set up in theprocedure room 50 where the patient is located, and in a control room52. The control room 52 is preferably adjacent the procedure room 50,and there may be a window 54 between the control room and the procedureroom to permit direct observation of the patient, however the controlroom could be remote from the patient, and with the aid of the presentinterface, a physician could conduct a procedure on a patient in theprocedure from a control room on a different floor, in a differentbuilding, or even in a different city.

[0033] The magnetic surgery system comprises a patient bed 56, and amagnetic navigation system 58 comprising opposed magnet units 60 and 62on opposite sides of the patient bed operated by a processor 64 andcontrolled by controls 66 adjacent the patient 56. An imaging system 68,such as an x-ray imaging system on a C-arm, displays images of theoperating region on a set of monitors 70 in the procedure room 50. Theinterface system of the present invention provides a convenient way fora user to operate the magnetic navigation system 58 to control thedistal end of a medical device in the operating region inside thepatient's body.

[0034] The interface includes a display on, for example, an lcd monitor72, and a digital tablet 74 in the procedure room 50, a processor 76, adisplay on, for example, monitor 78, a key board 80, and a mouse/digitaltablet 82 in the control room 54. Additional displays on monitors 86 and88 can be provided in the procedure room 50 which integrate images fromthe imaging system 68 with the interface. One or more additionalmonitors 90 can be provided in the control room so that the images areavailable in the control room as well. The monitor 90 preferablydisplays a multi-pane display.

[0035] In one preferred embodiment of this invention as shown in FIG. 2,one of the displays, for example display 90 has panes 92 and 94 forreceiving biplane images of the operating region. The display 90preferably also includes a control pane 96. As shown in FIG. 2, the userselects a discrete set of points on the distal end of the medical deviceto characterize the device. In the preferred embodiment, the operatingregion is imaged with a bi-planar imaging (such as bi-planefluoroscopy), providing two images of the operating region and of thedistal end of the medical device, in different, and preferably mutuallyperpendicular, planes. For example, the bi-plane imaging system mightprovide left anterior oblique (LAO) and right anterior oblique (RAO)images of the operating region, in panes 92 and 94, respectively.Bi-plane imaging could be provided with a single x-ray source andimaging plate that are moved in tandem to provide imaging in multipleplanes.

[0036] The user preferably identifies the “support” point or “pivot”point xi of the distal end portion of the medical device on each of thepanes 92 and 94. By identifying the point in both planes, the user hasuniquely identified the point in three dimensional space. The userpreferably also identifies the distal end x₄ of the medical device. Theuser preferably also identifies at least one, and preferably at leasttwo other points x₂ and x₃, between points x₁ and x₄. The user could ofcourse identify more points, but the additional accuracy achievedusually does not outweigh the inconvenience to the user.

[0037] Other methods of reconstructing the shape of the distal end ofthe medical device are possible. For example, points could be identifiedindependently on each of the views. Based upon the these points theprocessor could develop splines (a sequence of polynomial curves) foreach plane, and then identify corresponding points on the two splines tocharacterize the shape of the medical device in three dimensions.

[0038] As shown in FIG. 4A, the points can be identified by manipulatinga cursor on each of the bi-planar displays, and clicking on a point,such as point 100. As shown in FIG. 4B, after the user clicks on point100 on the LAO, a line 102 is displayed on the RAO, along which thepoint 100 identified in the LAO must lie. As shown in FIG. 4C, the userthen uniquely identifies the point in three dimensional space byidentifying the desired point 104 along the line 102 displayed on theRAO. The user could begin on the RAO, and complete the pointidentification process in the LAO. Some other method can be used toidentify points, or the points can be identified automatically, forexample using image processing.

[0039] Of course, instead of using bi-plane images to uniquely identifypoints on the medical device to characterize the shape of the medicaldevice, a localization system could be employed to characterize theshape of the medical device. For example a magnetic localization systememploying reference transmitters in the procedure room can transmit toone or more receivers on the medical device to locate the receiver andthus the medical device in three-dimensional space (or one or moretransmitters on the medical device can transmit to reference receiversin the procedure room). Other localization systems, for example usingultrasound, or electric potential, could be used.

[0040] Once the points x₁, x₂, x₃, and x₄ are identified inthree-dimensional space, the points can be processed to determine theconfiguration of the distal end of the medical device. This processingcan determine whether the point xi is in fact the support point andpivot point of the medical device. For example in a uniform medicaldevice, such as a uniform wall catheter, the distal end portion of themedical device, i.e. the portion distal to the pivot point will assume agenerally circular shape. Thus, whether a selected point xi is in factthe support point or pivot point can be determined by checking thecircularity of the reconstructed curve between the points x₁, x₂, x₃,and x₄. With other catheter configurations in which the properties arenot uniform along the length, the validity of x₁ as the support point orpivot point can be determined by checking the shape of the distal endportion against a calculated or empirically determined shape.

[0041] In this preferred embodiment, if the point x₁ selected as thetentative pivot point or support point, is determined not to be a validpivot point or support point, the fact is signaled to the user, forexample with a text message or by changing the color of the point on thedisplay to signal the user to select another tentative pivot point orsupport point. Through appropriate processing the system can evensuggest one or more appropriate pivot or support points for the user tosimply accept. Once the pivot or support point is correctly determined,the processor can then determine the free length l of the medical devicethat is distal to the pivot or support point.

[0042] With the pivot or support point x₁, the free length l, and theproperties of the distal end portion, as indicated by the positions ofthe points x₁, x₂, x₃, and x₄ identified by the user when a knownmagnetic field was applied, it is possible to calculate theconfiguration of the distal end portion of the medical device when adifferent magnetic field is applied. These calculations can take intoaccount the properties of the medical device as represented in a lookuptable or in one or more equations, developed by mathematical modeling orexperimental measurements. These calculations can also take into accountprevious sets of points x₁, x₂, x₃, and x₄ identified by the user forthe same (or even similar) medical devices.

[0043] Thus, it is possible to display a representation 106 of a virtualmedical device representing the configuration (shape and orientation) ofthe actual device if a different magnetic field were applied. (Ofcourse, for other remotely controllable medical devices, the interfacewould display a representation of a virtual medical device representingthe configuration of the actual device as if the control parameter(s)were changed as selected by the user). This representation can besuperimposed over the images of the operating region in panes 92 and 94.Thus, the user can select a desired new magnetic field, and then see theconfiguration of the distal end of the medical device as if the newfield were applied, before the new field is applied, and makeappropriate adjustments. A variety of systems and methods have beendevised for identifying the desired direction of the applied magneticfield. For example, the user can identify the starting point and theending point (for example on displays of a bi-plane imaging system), andthe field can be applied in the selected direction. Alternatively, theuser could manipulate a vector representation of the desired fielddirection, or select previously used directions, or select directionsassociated with previously identified points (with other remotelycontrollable medical devices the user can manipulate appropriate controlparameters to change the configuration of the device). However, in eachcase the resulting configuration of the medical device was merely aconjecture of the user, based upon experience. With the present method adisplay of the configuration that the distal end of the medical deviceshould assume when the desired field is applied can be made before thefield is actually applied.

[0044] Furthermore, as shown in FIG. 3A, it is possible to generate asurface 108 representing all the possible points that the distal end ofthe medical device can reach with only changes to one or more selectedcontrol parameters, such as the magnetic field direction and/orintensity. In this preferred embodiment, this surface 108 has agenerally parabolic shape. This surface 108 can also be displayed, andthe user can identify a point on this surface, and through processingthe system can determine the correct field to apply to cause the distalend of the medical device to reach the selected point. The user canorient the device in the desired direction and reach a point slightlybeyond the surface by advancing the medical device slightly. As shown inFIG. 3B, the system can even generate and display surfaces 108′ and 108″of possible points for different free lengths l′ and l″, so that if thesurface representing the current set of possible points does not reachthe desired location, the user can select from a surface of a differentset of possible points, and either manually adjust the free length l, orif the system is equipped with a mechanized advancer mechanism, allowthe system to automatically adjust the free length l.

[0045] The system and method of this invention also allow the user tochange the configuration of the displayed representation 106 of themedical device, and apply the field to achieve the desiredconfiguration. Thus the display can be coupled with controls that changethe configuration of the displayed virtual image 106 of the distal endportion of the medical device. These controls, can be for example,controls that change the deflection and rotation of the distal end ofthe device. As shown in FIG. 5, these controls may be implemented byscreen controls 110 and 112, which can be operated by pointing, clickingand dragging the control to change the degree of deflection and tochange the degree of rotation. These controls 110 and 112 are similar tothe controls provided on conventional mechanically navigable devices. Infact, rather than being implemented on the screen, the controls could beprovided in a catheter handle, similar to conventional catheter handles,which the user can manipulate to change the deflection and rotation ofthe distal end of the medical device. Once a desired deflection androtation are achieved using controls 110 and 112, as indicated by thedisplay of the virtual device in panes 92 and 94, the user can apply theappropriate field, for example by pointing and clicking on a virtualbutton 114 (or other control), to apply the magnetic field.Alternatively, the system can operate in a continuous mode, in which theconfiguration of the distal end portion of the medical automaticallychanges upon a change in the displayed image 106, without the need tooperate button 114). The possible configurations of the displayedvirtual device are preferably limited to configurations which can beachieved with possible applied magnetic fields, so that user cannotmanipulate the virtual device into a configuration that cannot beachieved. This could be built into the controls as well, so that thecontrols will not allow the user to manipulate the image 106 of thevirtual device into a configuration that cannot be physically achieved.

[0046] Just as the processor can determine the configuration for a givenapplied field, the processor can determine the appropriate applied fieldfor a given configuration. Thus the user can specify a configuration, ormore preferably a target point, and the interface can determine thecontrol parameters (e.g., magnetic field direction, magnetic fieldstrength, and free length) to reach the target. The interface displaysthe hypothetical new configuration and if satisfactory, the user canaccept it, and the interface-determined control parameter applied toreach the target.

[0047] The system can store the properties of various types of medicaldevices, and one of these stored values could be used in themathematical model that determines the shape of the display 106 of thevirtual medical device. The user can select some of these stored values,and as shown in FIG. 5, the pane 96 can also include a window 116 todisplay indicia identifying the stored values being used. The pane canalso include displays 118 of the magnetic field direction currentlybeing applied.

[0048] The interface employs some model of the medical device. Thefollowing description illustrates a model in the simple case of aflexible device with uniform elastic properties and with a singlepermanent magnet located at the device tip. The model and detailsgeneralize in a manner that may be determined by those skilled in theart to the case when non-uniform elastic properties and more than onemagnet are employed, or when alternative modes of device actuation areused. Mathematically, the configuration of the medical device can berepresented as follows:

[0049] The distances between the points x₁, x₂, x₃, and x₄, are given asfollows: $\begin{matrix}{d_{1} = {{{\overset{}{x}}_{2} - {\overset{}{x}}_{1}}}} & \lbrack 1\rbrack \\{d_{2} = {{{\overset{}{x}}_{3} - {\overset{}{x}}_{2}}}} & \lbrack 2\rbrack \\{d_{3} = {{{\overset{}{x}}_{4} - {\overset{}{x}}_{3}}}} & \lbrack 3\rbrack\end{matrix}$

[0050] The unit tangent vectors at points x₁ and x₄ ({right arrow over(n)}₁ and {right arrow over (n)}₄) may be estimated as follows:$\begin{matrix}{{\overset{}{t}}_{1} = \frac{\left( {{\overset{}{x}}_{2} - {\overset{}{x}}_{1}} \right)}{d_{1}}} & \lbrack 4\rbrack \\{{\overset{}{t}}_{2} = \frac{\left( {{\overset{}{x}}_{3} - {\overset{}{x}}_{2}} \right)}{d_{2}}} & \lbrack 5\rbrack \\{{\overset{}{t}}_{3} = \frac{\left( {{\overset{}{x}}_{4} - {\overset{}{x}}_{3}} \right)}{d_{3}}} & \lbrack 6\rbrack \\{{\overset{}{n}}_{4}^{\prime} = {{\overset{}{t}}_{3} + {\frac{\left( {{\overset{}{t}}_{3} - {\overset{}{t}}_{2}} \right)}{d_{2}} \cdot d_{3}}}} & \lbrack 7\rbrack \\{n_{4} = \frac{{\overset{}{n}}_{4}^{\prime}}{{\overset{}{n}}_{4}^{\prime}}} & \lbrack 8\rbrack \\{{\overset{}{n}}_{1}^{\prime} = {{\overset{}{t}}_{2} + {\frac{\left( {{\overset{}{t}}_{3} - {\overset{}{t}}_{2}} \right)}{d_{2}} \cdot d_{1}}}} & \lbrack 9\rbrack \\{n_{1} = \frac{{\overset{}{n}}_{1}^{\prime}}{{\overset{}{n}}_{1}^{\prime}}} & \lbrack 10\rbrack\end{matrix}$

[0051] θ, the deformation angle between x₁ and x₄, can be found from cosθ={right arrow over (n)}₁·{right arrow over (n)}₄.

[0052] Deviation from Planarity

[0053] The points x₁, x₂, x₃, and x₄ can be checked to make sure thatthey fall generally in a plane. {right arrow over (p)} the unit vectorbetween x₁ and x₄ is:

{right arrow over (p)}=({right arrow over (x)} ₄ −{right arrow over (x)}₁)/|{right arrow over (x)} ₄ −{right arrow over (x)} ₁|  [11]

[0054] Then {right arrow over (q)}₁, a vector orthogonal to {right arrowover (n)}₁ and {right arrow over (n)}₄, is given by:

{right arrow over (q)} ₁ ={right arrow over (n)} ₁ ×{right arrow over(n)} ₄   [12]

[0055] and {right arrow over (q)}₂, a vector orthogonal to {right arrowover (n)}₁ and {right arrow over (p)}, is given by:

{right arrow over (q)} ₂ ={right arrow over (n)} ₁ ×{right arrow over(p)}  [13]

[0056] The angle φ between {right arrow over (q)}₁ and {right arrow over(q)}₂ is a measure of the planarity of the points, is given by:

φ=cos⁻¹({right arrow over (q)} ₁ ·{right arrow over (q)} ₂)   [14]

[0057] For planarity it is desirable that φ≦10°≈π/20

[0058] Circularity Check (Uniform Stiffness)

[0059] For a medical device of uniform stiffness, the distal end portionwill generally assume a circular configuration. The selection of thepoints, and particularly the pivot point x₁ can be validated by ensuingthat they lie substantially along a circle. (For not uniform devices,some other check can be performed on the points). Letting {right arrowover (u)} represent the field direction unit vector and definingψ=cos⁻¹({right arrow over (u)}·{right arrow over (n)}₁), then α, the lagangle between the applied magnetic field and the direction of the isgiven by and define α≡(ψ−θ). If m is the magnetic moment of the magneton the distal end of the medical device, and β its stiffness (β=EI whereE is the Young's modulus of the material and I is the bending moment ofarea), then: $\begin{matrix}{l = {\frac{\beta}{mB} \cdot \frac{\theta}{\sin \quad \alpha}}} & \lbrack 15\rbrack\end{matrix}$

[0060] where B is the magnitude of the field strength and l is thelength of the medical device between {right arrow over (x)}₁ and {rightarrow over (x)}₄.

[0061] The chord length is given by: $\begin{matrix}{d^{\prime} = {\frac{2l}{\theta}\sin \frac{\theta}{2}}} & \lbrack 16\rbrack\end{matrix}$

[0062] If the points lie along a circle, the chord length d′ should beclose to c≡|{right arrow over (x)}₄−{right arrow over (x)}₁|, forexample, within ten percent, or$\frac{{c - d^{\prime}}}{c} \leq {0.1.}$

[0063] If the above constraints are satisfied, the curve of the medicaldevice can be estimated as follows:

[0064] v, the unit vector in the plane of the medical device orthogonalto {right arrow over (n)}₁:

{right arrow over (v)}′≡{right arrow over (n)} ₄ −{right arrow over (n)}₁ cos θ  [17]

[0065] The curve of the medical device is defined by: $\begin{matrix}{{\overset{}{x}}^{\prime} = {{\overset{}{x}}_{1} + {\frac{l}{\theta}\left( {1 - {\cos \quad \theta^{\prime}}} \right)\overset{}{v}} + {\frac{l}{\theta}\sin \quad \theta^{\prime}{\overset{}{n}}_{1}}}} & \lbrack 19\rbrack\end{matrix}$

[0066] for 0≦θ′≦θ

[0067] The envelope or surface of the tip as field direction varies canbe determined, based upon the following formula for tip position:$\begin{matrix}{{\overset{}{x}}_{tip} = {{\overset{}{x}}_{1} + {\frac{l}{\theta}\left( {1 - {\cos \quad \theta}} \right)\overset{}{v}} + {\frac{l}{\theta}\sin \quad \theta {\overset{}{n}}_{1}}}} & \lbrack 20\rbrack\end{matrix}$

[0068] Varying θ from 0 to θ_(mas) (=lMB/β) gives the envelope or locusof tip positions as the field is varied.

[0069] Determining the Field Direction for a Selected Point

[0070] If a point Z is selected on the envelope, the field directionthat take the tip to that location may be computed as follows:$\begin{matrix}{{\overset{}{u}}_{1} = \frac{\left( {\overset{}{z} - {\overset{}{x}}_{1}} \right)}{{\overset{}{z} - {\overset{}{x}}_{1}}}} & \lbrack 21\rbrack \\{\overset{\_}{\varphi} = {\cos^{- 1}\left( {{\overset{}{u}}_{1} \cdot {\overset{}{n}}_{1}} \right)}} & \lbrack 22\rbrack \\{\overset{\_}{\theta} = {{\cot^{- 1}\left\lbrack \frac{\left( {1 - {\tan^{2}\overset{\_}{\varphi}}} \right)}{2\quad \tan \quad \overset{\_}{\varphi}} \right\rbrack} = {2\overset{\_}{\varphi}}}} & \lbrack 23\rbrack \\{\overset{\_}{\psi} = {\overset{\_}{\theta} + {\sin^{- 1}\left( \frac{\beta \quad \overset{\_}{\theta}}{MBl} \right)}}} & \lbrack 24\rbrack\end{matrix}$

[0071] Then the corresponding field direction to orient the tip to pointz is given by the unit vector:

{right arrow over (u)} _(B) ={right arrow over (n)} ₁ cos {overscore(ψ)}+{right arrow over (v)} sin {overscore (ψ)}  [25]

[0072] An “accessible surface” may also be defined by rotating theenvelope obtained in equation 25 about {right arrow over (n)}₁. {rightarrow over (w)}, the unit vector orthogonal to {right arrow over (v)}and {right arrow over (n)}₁, is given by {right arrow over (w)}={rightarrow over (v)}×{right arrow over (n)}₁, and {right arrow over (v)}_(R),the rotation of {right arrow over (w)} about {right arrow over (n)}₁ isgiven by {right arrow over (v)}_(R)=cos ξ{right arrow over (v)}+sinξ{right arrow over (w)} for 0≦ξ≦2π

[0073] Then: $\begin{matrix}{x_{s} = {x_{1} + {\frac{l}{\theta}\left( {1 - {\cos \quad \theta}} \right)\left( {{\cos \quad \xi \overset{}{v}} + {\sin \quad \xi \overset{}{w}}} \right)} + {\frac{l}{\theta}\sin \quad \theta {\overset{}{n}}_{1}}}} & \lbrack 26\rbrack\end{matrix}$

[0074] defines a surface of revolution about n₁ that is the surfaceaccessible to the tip by changing the field direction by varying θ andξ, a surface is generated analogous to generating a sphere withlongitude and latitude.

[0075] Generating the Field Direction to any Point on the Surface

[0076] As in Equation [25], given a desired point {right arrow over (z)}on this surface, a corresponding field direction is obtained from:

u _(B,S) ={right arrow over (n)} ₁ cos ψ+{right arrow over (v)}_(R,S)sin {right arrow over (ψ)}  [27]

[0077] where {right arrow over (v)}_(R,S) is obtained from:

{right arrow over (v)}_(R,S)=cos ξ_(s) {right arrow over (v)}+sinξ_(s){right arrow over (w)}  [28]

[0078] where $\begin{matrix}{\xi_{s} = {\tan^{- 1}\left( \frac{\overset{}{u} \cdot \overset{}{w}}{\overset{}{u} \cdot \overset{}{v}} \right)}} & \lbrack 29\rbrack\end{matrix}$

[0079] When the desired tip location is beyond the accessible surface,then a combination of change of inserted length and field direction isneeded to move the distal end of the device to {right arrow over (z)}from its current location.

[0080] As before, define {right arrow over (u)}₁, and find {overscore(φ)} and {overscore (θ)}. If {overscore (l)} is the new medical devicelength, and $\begin{matrix}{{y = {\left( {\overset{\rightarrow}{z} - {\overset{\rightarrow}{x}}_{1}} \right) \cdot {\overset{\rightarrow}{n}}_{1}}},{{{then}\quad \overset{\_}{l}\frac{\sin \overset{\_}{\theta}}{\overset{\_}{\theta}}} = {{y\quad {or}\quad \overset{\_}{l}} = {{\frac{y\overset{\_}{\theta}}{\sin \quad \overset{\_}{\theta}}.\overset{\_}{\psi}} \equiv {\overset{\_}{\theta} + {\sin^{- 1}\left( \frac{\beta \quad \overset{\_}{\theta}}{{MB}\overset{\_}{l}} \right)}}}}}} & \lbrack 30\rbrack\end{matrix}$

[0081] Given Equations 28 and 29:

{right arrow over (u)} _(B,S) ={right arrow over (n)} ₁ cos {overscore(ψ)}+{right arrow over (v)} _(R,S) sin {overscore (ψ)}  [31]

[0082] and

δl=({overscore (l)}−l)   [32]

[0083] is the change in inserted length which together with the changein field direction will take the end of the medical device to thedesired location.

[0084] Of course, while described herein in the context of controlling amagnetic surgery system, as stated earlier the virtual device interfaceof this invention could be applied to other systems which can controlthe configuration of medical device, including devices whoseconfiguration is controlled mechanically, hydraulically, or throughmagnetostrictive and electrostrictive means.

[0085] A second embodiment of user interface is illustrated in FIG. 10.The interface of the second embodiment is adapted to help a user controla magnetic navigation system that applies a magnetic field to anoperating region in a patient to control the direction of a magneticmedical device in the operating region. An advancer/retractor mechanismis preferably also provided for the controlled advancement/retraction ofthe medical device. As shown in FIG. 10, the user is presented aninterface with several virtual or real buttons, which the user canoperate using a mouse, track ball, space ball, joy stick, tablet, orother input device. Operating button 200 initiates the navigation underthe present invention, and a window (or other message) is displayed,prompting the user to use the input device to identify a number ofpoints on the medical device on the display 202, including an image 203of the medical device including the anchor point.

[0086] The user identifies points, for example, by positioning thecursor over the medical device on the display 202. In this preferredembodiment, the inventors have determined that four points is anappropriate balance between user effort and accuracy in characterizingthe medical device to be navigated, but other numbers of points couldalso be used. The points identified are checked or qualified (asdescribed above) and a window (or other message) alerts the user if oneor more points are not properly identified.

[0087] With information about the currently applied magnetic field andthe position and configuration of the medical device (resulting from theidentification of points), the processor implementing the interface cancharacterize the medical device. The user can then use the input deviceto operate direction control buttons. In this preferred embodiment thereare four such buttons, buttons 204 and 206 for increasing and decreasingthe deflection of the medical device, and buttons 208 and 210 forrotating the medical device. The interface can be operated in a discretenavigation mode, a continuous navigation mode, or the interface canallow the user to switch between discrete and continuous navigation. Inthe discrete navigation mode the user clicks on the buttons 204, 206,208, or 210, and the image 212 of the a virtual device is superposedover the display 202 showing the configuration of the medical devicewith the specified new magnetic field applied. Once the user issatisfied with the new field, as represented by the image 212 of thevirtual device, then the user operates a control (such as a button) toactivate the magnetic navigation system and apply the specified field,causing the actual medical device to assume substantially theconfiguration and position represented by the image 212 of the virtualdevice. In the continuous navigation mode, the user could operate in acontinuous navigation mode, changes in the direction specified byoperating buttons 204, 206, 208, and 210 are automatically implemented,moving the medical device. The user may enter the continuous navigationmode, for example, by holding down a control button. The buttons 204,206, 208, and 210 preferably can be operated to change deflection androtation in predetermined, discrete amounts, which preferably can becustomized by the user, or can be held down to continuously change thedeflection and rotation to the limits of the navigation system. Manyother mappings besides separate control of rotation and deflection ofthe device are possible. For example, the device tip could be controlledor actuated suitably to move within a chosen plane.

[0088] In another embodiment a joystick is provided for interactivedevice control purposes. In this case, the user would select from a setof possible mappings from joystick deflections to changes in actuationcontrol variables that would modify the configuration of the device. Inone particular case where a magnetic surgery system is used for deviceactuation, the magnetic field would be driven from the joystick. Visualfeedback from an imaging system that could employ X-ray, MagneticResonance Imaging, Ultrasound or other imaging modalities known topractitioners of the art provides the user with the deviceconfiguration, so that the device may be driven interactively by theuser to reach a desired target. Other embodiments may use spaceballs ora variety of other input devices, including those that are custom-built,for interactive remote control of device actuation.

[0089] The interface also allows the user to click on a point in thedisplay to mark a target point 214 on the two dimensional display. Theinterface then determines the magnetic field required of the magneticnavigation system and extension or retraction of the medical device(which is preferably controlled by an automated advancing/retractingsystem) required to reach the target point 214, and displays an image216 of a virtual device with the calculated magnetic field applied, andwith the appropriate adjustment in length made. The user then operates acontrol (such as a button), so that calculated field and adjustments inlength are applied. Alternatively, the user could operate in acontinuous navigation mode (such as by holding down another button on ajoystick) so that the applied magnetic field and device lengthautomatically change to bring the distal end of the medical device tothe point 214 identified by clicking on the display 202.

[0090] Because the user is attempting to identify a point in threedimensional space by identifying a point on a two dimensional display,the distal end of the medical device may not be precisely where the userintended. The user can switch to another display (if biplane imaging ofthe operating region is available) to refine the position.Alternatively, where biplane imaging or two corresponding images areavailable, the user can identify the target point on each image,uniquely identifying the point in three dimensional space. After theuser identifies the target in one image, the interface assists theselection of a proper point in the second image. For example, the cursorcolor or shape can indicate when the cursor is over a valid or invalidposition based upon the user's selection on the other display. Indiscrete navigation, the magnetic navigation system applies the requiredfield to achieve the specified target point when the user actuates anapply field button. In one embodiment of continuous navigation, themagnetic navigation system automatically begins to apply the requiredfield to achieve the specified target point as soon as the target pointis properly specified. To prevent unintended movement, the userpreferably has to hold down a button to remain in the continuousnavigation mode, and when the button is released the magnetic navigationsystem ceases movement. In an embodiment that employs a joystick, forexample, the button could be a trigger button on the joystick. Inpractice, the apply field button used in the discrete navigation modeand the continuous navigation button used in the continuous navigationmode can be the same.

[0091] Alternatively, the user can make minor adjustments to theposition using the buttons 204, 206, 208, and 210. Once the medicaldevice is characterized, as prompted by the interface system, the systemcan automatically interpret further pointing and clicking on the display202 as an indication that the user is identifying a target point 214.Alternatively, a button can be provided on the display or on the inputdevice for the user to turn on the target mode.

[0092] As mentioned previously, rather than buttons 204, 206, 208, and210, the user can input directional changes with a joystick. Forexample, moving the joystick forward and backward can increase anddecrease the deflection, and moving the joystick left and right (ortwisting the joystick) can cause rotation. Likewise, it is possible toemploy other modes of mapping from the joystick to device actuation. Aswith using the buttons above, the user can operate in discrete orcontinuous mode. When operating in discrete mode, the user uses thejoystick to position a virtual image 212 on the display 202, and whensatisfied with the position, operates a control (such as a button on thejoy stick) to cause the navigation system to apply the magnetic field.When operating in the continuous mode, the user operates a control (suchas holding down a button on the joystick) so that changes indicated bythe movement of the joystick are automatically implemented, with themagnetic navigation system moving in response to change the fields asrequired. In this mode the medical device responds interactively as theuser operates the joystick.

[0093] In a preferred embodiment, advancement of the medical device ispreferably separately controlled by the user, for example with a toggleswitch on the joystick so that a single joystick can be used for bothdevice steering and device advancement or retraction, or with a separatebutton on the display 202. However in the target mode, the interfacepreferably controls both the magnetic navigation system and the advancersystem, so that the device is directly computer controlled. In yetanother preferred embodiment, advancement of the medical device anddevice deflections or shape changes are controlled by the user fromseparate joysticks. Various other embodiments and modes of use may beconceived by those skilled in the art.

[0094]FIG. 11 shows a display 300 from another implementation of theinterface system of the second embodiment. The display 300 includes twobi-plane images 302 and 304 of the operating region. The display alsoincludes a control pane 306. The control pane may include a box 308 inwhich the user can select or specify the stiffness of the medicaldevice; alternatively the medical device type may be selected from amenu of medical devices whose stiffness value is pre-programmed into thesystem. A computer processor can use the stiffness to determine theconfiguration of the medical device in an applied magnetic field, bothto generate virtual images of the device under specified magneticfields, and to calculate the applied field to reach a particular targetpoint.

[0095] The control pane 306 also includes a field display 309, with thecoordinates of the current magnetic field direction, and pick boxes forselecting the point of application of the field. The control pane hasfree length box 310, which allows the user to select the free length ofthe medical device from the end of its sheath. Arrows 312 and 314 allowthe user to increase and decrease the specified length, which isimplemented by an advancer/retractor mechanism. The control pane 306also includes slide controls 316 and 318 for controlling the degree ofdeflection and rotation (like buttons 204 206, 208 and 210). An applyfield button 320 allows the user to apply the field specified by thevirtual device in the displays. A conform to field button 322 causes thevirtual device model to predict and show the shape that would be causedby application of an input magnetic field. A get live button 324 updatesthe images 302 and 304. Finally, a target mode button 326 allows theuser to enter the target mode (as described above), so that pointing andclicking on the displays 302 and 304 identifies a point which thevirtual medical device moves to in the discrete mode, or which theactual medical device moves in the continuous mode.

[0096] In still another embodiment of user interface according to theprinciples of this invention, the user interaction can be simplifiedfurther for ease of use. In FIG. 12, graphical buttons 411 and 412 areprovided as alternate modes of control of a magnetic navigation system.Button 411 when selected enables defining the direction of the magneticfield in three dimensional space by specification of the direction intwo projections (commonly the LAO and RAO perspectives familiar tointerventional surgeons). When the corresponding magnetic field isapplied to steer a medical device, the device tends to align itself withthe magnetic field but with a certain angular lag as dictated by theelastic and magnetic properties of the device. In contrast, button 412enables direct selection of a target location that the device can moveto. In order to use this mode, the user first selects a “Mark catheterbase” button 413 (shown highlighted in FIG. 12) with a mouse click. Thisenables the user to indicate (by drawing with a pen-tablet or bysuitably dragging a mouse) the pivot or point of support of the deviceas well as the pivot direction, by suitably drawing a line in twoprojected X-ray views. In FIG. 12, a catheter 419 can be seen to extendfrom a guide sheath 422. The line 420 drawn by the user indicates thepivot point to be the distal tip of the sheath and the direction inwhich the device 419 is extended at the pivot point. FIG. 13 shows thepivot point and direction marked by lines 505 and 506 respectively inRAO and LAO views 501 and 502. This process in effect provides x₁ and n₁to the system. Next the user selects the target button 412 as shown inFIG. 14. This provides to the user a “Target” cursor 601 that enablesselection of a point 602 in three dimensional space from two X-rayprojections. This point is the desired anatomical target point that theuser wishes to steer the medical device to. The computer then calculatesthe requisite field direction and insertion length from equations (25)and (32). The corresponding shape of the device is also computed anddisplayed as a dashed curve 603. FIG. 15 shows a desired target point inRAO and LAO projections respectively as 602 and 602 a together withassociated projections of the device shape 603 and 603 a 11respectively.

[0097] The required field direction and insertion length may beautomatically applied by the system upon the press of a button.Alternatively, the field direction may be applied directly by the systemwhile the user continuously controls the advancement or retraction ofthe device. The device curve may be updated to be shown as a solidcurve, perhaps in a different color, to indicate that the desired changeof controls has been effected. FIG. 16 shows an LAO perspective of suchan updated device curve 650 together with the user-defined pivotdirection 648 and the desired target location 652. Further the deviceshape may be interactively manipulated by the user as described above.One convenient implementation uses keyboard buttons for increasing ordecreasing deflection of the virtual device and for rotating itclockwise or anti-clockwise about the pivot direction. The device shapeis continuously computed and updated in the projected displays as wellas possibly in a three dimensional display during this process of useradjustment. The corresponding manipulated virtual device 702 is shown inan LAO perspective in FIG. 17 together with the current deviceconfiguration 701 and an updated location 703 for the device tip as theuse manipulates the virtual device. Once a desired device shape or tiplocation is achieved the corresponding controls may be applied at thetouch of an “Apply” button, or alternatively the system may apply thecontrols (magnetic field and insertion/retraction in one embodiment) inuser-defined steps every time a keyboard button is pressed. Anotherconvenient implementation of continuous control uses a joystick or otherinput device together with a mapping function to convert joystickmovements to changes in control inputs, causing the device to respondcontinuously to joystick movements. In one implementation the usercould, together with moving or deflecting the joystick, press and hold a“trigger” button to cause the changes in control inputs to be applied.The user observes (on an X-ray image or other mode of visualization) thedevice response to joystick movements and may interactively stop deviceresponse when a desired device configuration has been attained, forexample by releasing a “trigger” button on the joystick.

[0098] The actuation system employed to steer or orient the flexibledevice can take several different forms such as magnetic actuation byexternal magnetic fields, mechanical actuation through the use ofsteering cables or other force transmission elements within the device,device actuation using piezoelectric materials or electrostrictivepolymers such as silicone elastomers, actuation with magnetostrictiveelements embedded in the device, or other technologies known topractitioners of the art.

[0099] In a preferred embodiment, the actuation system can use magneticfields generated by external magnets to orient magnets incorporated inthe distal portion of the device. The interaction of the external fieldwith the device magnets serves to orient the device in controllablefashion by controlling the externally applied magnetic field. Theexternal magnets could employ either permanent magnets or electromagnetspossibly employing superconductivity to produce strong magnetic fields.FIG. 1 shows a schematic illustration of such a system and device.

[0100] In an alternate preferred embodiment employing electrostrictiveactuation, the device could use electrically activated electrostrictivematerials to bend or steer the device. FIG. 18 shows a flexible device851 which incorporates electrostrictive elements 854 and 855. Electricalleads 858, 859 and 865, 866 connect the elements 854 and 855respectively to a voltage source 856. The electrostrictive element 854is sandwiched between rings 860 and 862. When a voltage is appliedacross the leads 858 and 859, the electrostrictive element 854 decreasesin length, forcing the flexible device to bend in the plane defined bythe device axis and the orientation of the electrostrictive elementrelative to the device axis. The amount of bending can be controlled byvarying the applied voltage. When a voltage is applied across leads 865and 866, the shortening of element 855 forces the device to bend in theopposite direction.

[0101] In some cases, it may be useful to determine the bending plane ofthe device. This is relevant where a non-magnetic steering mechanism isused to orient the device. This may be done by employing an imagingmethod such as X-ray imaging or by using a localization sensor embeddedin the device. If a localization sensor which gives complete positionand orientation of the device is used, the orientation of the actuationmechanism such as steering cable or electrostrictive element is known ina reference configuration, the corresponding orientation in a generalconfiguration may be obtained by processing the data obtained from thelocalization sensor. Thus the bending plane of the device in threedimensional space may be determined. Alternatively, from imaging such asfluoroscopy, the change in the tangent vector to the device tip upon asmall change in an actuation control variable may be recorded, whichalso provides information needed to determine the bending plane in threedimensional space.

[0102] Knowledge of the bending plane can be used to control the devicefor targeting purposes and for configurational changes with the aid of acomputational model in a manner similar to that described above.

[0103] The embodiments discussed here are for purposes of example only,and other embodiments of actuation systems such as mechanical actuatorsor others known in the field can be used for device steering. In thefollowing a computational model for magnetic actuation with externallyapplied magnetic fields is described as a non-limiting example, with theunderstanding that the computational model and its use can generallyincorporate a variety of possible actuation systems.

[0104] The actuation system coordinates may be registered to the imagingsystem coordinates and to the localization system coordinates by meansof several methods known to those skilled in the art. For example, theactuation system may employ external magnetic fields, in which case thesource of the fields such as an external magnet could be mechanicallyregistered to the imaging system by the use of suitably placed markersplaced in known positions with respect to the actuation system. Thesepositions may also be ascribed coordinates in the reference frame of theimaging system since information from multiple imaging projections orfrom three dimensional imaging would be available from the imagingsystem. The positions of three markers determined in both coordinatesystems suffices to generate a rigid coordinate transformation (rotationand translation) which is the requisite registration. Additional markersmay be used for improved accuracy as desired.

[0105] Likewise if a device localization system is used, it could beused to determine the coordinates of known markers with respect to thelocalization system as well, thence providing a mutual registration ofthe various distinct coordinate frames employed by the various systems.In practice, such a registration would be performed at the start of amedical procedure. Subsequent movement of the patient table may betracked to within a suitable accuracy to permit updated registrations asmay be required on a regular basis during a clinical procedure.

[0106] While a magnetic actuation system has been discussed in the abovefor non-limiting illustrative purposes, various alternative actuationsystems may be deployed and registered as needed according to themethods taught here. For example, a device employing piezoelectricactuators may use a coordinate frame in common with an X-ray imagingsystem, requiring only registration to a device localization system.Likewise, the markers in question may be either mechanically placed bythe user or may be based on anatomical locations as pinpointed by theuser.

What is claimed is:
 1. A method of operating a navigation system thatremotely configures the distal end portion of a medical device inside asubject's body, the method comprising: displaying a virtualrepresentation of at least a portion of the medical device, based upon acomputational model of the medical device; accepting inputs made by auser to change the configuration of the portion of the medical devicerepresented by the displayed virtual representation, and updating thevirtual representation of the portion of the medical device; operatingthe navigation system using the computational model of the medicaldevice to cause the portion of the medical device to conform to thedesired configuration represented by the updated virtual representation.2. The method according to claim 1 wherein the user makes inputsremotely from the subject.
 3. The method according to claim 1 whereinthe medical device is an elongate medical device having a proximal endoutside the subject's body, and a distal end inside the subject's body,wherein the virtual representation is of at least the distal end portionof the medical device, and wherein the navigation system changes theconfiguration of at least the distal end portion of the medical device.4. The method according to claim 3 wherein the step of displaying avirtual representation of the distal end portion of the elongate medicaldevice includes superposing the representation of the end portion of theelongate medical device over a representation of the operating region inthe subject.
 5. The method according to claim 4 wherein therepresentation of the operating region in the patient is derived from anx-ray image.
 6. The method according to claim 4 wherein therepresentation of the operating region in the subject is derived from anCT image.
 7. The method according to claim 4 wherein the representationof the operating region in the subject is derived from an MR image. 8.The method according to claim 4 wherein the representation of theoperating region in the subject is derived from an ultrasound image. 9.The method according to claim 4 wherein the representation of theoperating region in the subject is derived from a map of points onsurfaces in the operating region.
 10. The method according to claim 3wherein the navigation system operates in response to control variables,and wherein the step of accepting inputs made by a user to change theconfiguration of the distal end portion of the elongate medical devicerepresented by the displayed virtual representation, comprises acceptingdesired changes to the control variables of the navigation system,corresponding to desired changes in the operating state of thenavigation system, and updating the virtual representation of the distalend portion of the elongate medical device to show the medical device asif the control variables of the navigation system were actually changed.11. The method according to claim 10 wherein the navigation systemcomprises at least one external magnet for applying a magnetic field toa magnetically responsive element on the medical device, and wherein thesystem accepts inputs of desired changes in the magnetic fielddirection.
 12. The method according to claim 3 wherein the step ofaccepting inputs made by a user to change the configuration of thedistal end portion of the elongate medical device represented by thedisplayed virtual representation, includes accepting input of a newdirection for the distal end portion of the elongate medical device. 13.The method according to claim 3 wherein the step of accepting inputsmade by a user to change the configuration of the distal end portion ofthe elongate medical device represented by the displayed virtualrepresentation, includes accepting input of a deflection for the distalend portion of the elongate medical device from its currentconfiguration.
 14. The method according to claim 3 wherein acceptinginputs made by a user to change the configuration of the distal endportion of the elongate medical device represented by the displayedvirtual representation, includes accepting input of a new position forthe distal end portion of the elongate medical device.
 15. The methodaccording to claim 3 wherein the navigation system comprises elements inthe medical device which, when operated, change the configuration of thedistal end portion of the medical device.
 16. The method according toclaim 15 wherein at least some of the elements which change theconfiguration of the distal end portion of the medical device areelectrostrictive elements.
 17. The method according to claim 15 whereinat least some of the elements which change the configuration of thedistal end portion of the medical device are magnetostrictive elements.18. The method according to claim 17 further comprising applying amagnetic field from an external source magnet to the magnetostrictiveelements change the configuration of the distal end portion of themedical device.
 19. The method according to claim 3 wherein thenavigation system comprises mechanical links in the medical devicewhich, when translated, change the configuration of the distal endportion of the medical device.
 20. The method according to claim 3wherein the step of operating the navigation system occurs automaticallyafter the inputs by the user are accepted.
 21. The method according toclaim 3 wherein the step of operating the navigation system only occursafter an input by the user.
 22. The method according to claim 3 whereinthe computational model is based at least in part on the physicalcharacteristics of the medical device.
 23. The method according to claim22 wherein the computational model is based at least in part on amathematical representation of at least one physical characteristic ofthe medical device.
 24. The method according to claim 22 wherein thecomputational model is based at least in part on measurements of atleast one physical characteristic of the medical device.
 25. The methodaccording to claim 3 wherein the position of the distal end portion ofthe medical device is controlled by an advancing and retractingmechanism, and wherein the method further comprises accepting inputsmade by the user to change the position of the distal end portion of themedical device, and updating the virtual representation of the distalend portion of the elongate medical device; and operating the advancingand retracting mechanism to cause the distal end portion of the elongatemedical device to conform to the desired position represented by theupdated virtual representation.
 26. The method according to claim 3further comprising determining the actual position and configuration ofthe distal end portion of the medical device, comparing the actualposition and configuration of the distal end portion of the medicaldevice, with the position and configuration given by a computationalmodel to determine the force of contact between the medical device andtissue in the operating region by computational means.
 27. A method ofoperating a navigation system that remotely configures the distal endportion of a flexible medical device inside a subject's body to reach atarget, the method comprising; displaying a representation of theoperating region; accepting an input of the configuration of a portionof the device and displaying a representation of the configuration ofsaid portion; accepting an input of a target on the display of theoperation region; determining the inputs necessary to the navigationsystem to cause the medical device to reach the target based upon acomputational model of the medical device and the navigation system;applying the determined inputs to the navigational system to cause themedical device to reach the target.
 28. The method according to claim 27wherein the medical device is an elongate medical device having aproximal end outside the subject's body, and a distal end inside thesubject's body.
 29. The method according to claim 28 wherein thenavigation system can both change the configuration of the distal endportion of the medical device and advance or retract the distal end ofthe elongate medical device.
 30. The method according to claim 28wherein the navigation system comprises at least one magnet that appliesa magnetic field of selected direction to a magnetically responsiveelement on the medical device, and wherein determining the inputsnecessary to the navigation system to cause the elongate medical deviceto reach the target based upon a computational model of the elongatemedical device and the navigation system includes determining themagnetic field direction needed to cause the medical device to reach thetarget based upon the computational model.
 31. The method according toclaim 30 herein determining the inputs necessary to the navigationsystem to cause the elongate medical device to reach the target basedupon a computational model of the elongate medical device and thenavigation system includes determining the magnetic field strengthneeded to cause the medical device to reach the target based upon thecomputational model.
 32. The method according to claim 30 wherein theuser makes inputs remotely from the subject.
 33. The method according toclaim 30 wherein the step of displaying a virtual representation of thedistal end portion of the elongate medical device includes superposingthe representation of the end portion of the elongate medical deviceover a representation of the operating region in the subject.
 34. Themethod according to claim 33 wherein the representation of the operatingregion in the subject is derived from an x-ray image.
 35. The methodaccording to claim 33 wherein the representation of the operating regionin the subject is derived from an CT image.
 36. The method according toclaim 33 wherein the representation of the operating region in thesubject is derived from an MR image.
 37. The method according to claim33 wherein the representation of the operating region in the subject isderived from an ultrasound image.
 38. The method according to claim 33wherein the representation of the operating region in the subject isderived from a map of points on surfaces in the operating region. 39.The method according to claim 27 wherein the navigation system comprisesat least one external magnet for applying a magnetic field to amagnetically responsive element on the elongate medical device, andwherein the control inputs include changes in the magnetic fielddirection.
 40. The method according to claim 27 wherein the navigationsystem comprises elements in the medical device which, when operated bythe navigation system, change the configuration of the distal endportion of the medical device.
 41. The method according to claim 40wherein at least some of the elements which change the configuration ofthe distal end portion of the medical device are electrostrictiveelements.
 42. The method according to claim 40 wherein at least some ofthe elements which change the configuration of the distal end portion ofthe medical device are magnetostrictive elements.
 43. The methodaccording to claim 40 wherein the navigation system comprises mechanicallinks in the medical device which, when translated, change theconfiguration of the distal end portion of the medical device.
 44. Themethod according to claim 27 wherein the step of operating thenavigation system occurs automatically after the inputs by the user areaccepted.
 45. The method according to claim 27 wherein the step ofoperating the navigation system only occurs after an input by the user.46. The method according to claim 27 wherein the computational model isbased at least in part on the physical characteristics of the medicaldevice.
 47. The method according to claim 46 wherein the computationalmodel is based at least in part on a mathematical representation of atleast one physical characteristic of the medical device.
 48. The methodaccording to claim 46 wherein the computational model is based at leastin part on measurements of at least one physical characteristic of themedical device.
 49. A method of navigating the distal end of a medicaldevice having a magnetically responsive element through the applicationof a magnetic field from an external source magnet, the methodcomprising: accepting inputs from the user of a desired magnetic field;displaying a virtual representation of the configuration of the distalend of the medical device as if the desired magnetic field were applied;accepting an instruction from the user to apply the desired magneticfield.
 50. A method of navigating the distal end of a medical devicehaving a magnetically responsive element through the application of amagnetic field from an external source magnet, the method comprising:identifying a plurality of points on an image of the distal end portionof the medical device while a known magnetic field is applied to thedistal end portion of the medical device; processing the identifiedpoints to determine at least one geometrical characteristic of thedistal end of the medical device; allowing the user to input a desiredmagnetic field; displaying a virtual representation of the distal end ofthe medical device as if the desired magnetic field were applied; allowthe user to input an instruction to apply the desired magnetic field andreorient the medical device.
 51. A method of navigating the distal endof a flexible medical device having actuation elements controlled byapplying a set of actuation controls, the method comprising: displayinga virtual representation of the distal end portion of the medicaldevice; in response to inputs of desired changes in the shape andorientation of the distal end of the medical device that can be effectedby changing the applied actuation controls, updating the displayedvirtual representation of the distal end portion of the medical deviceby using a computational physics-based model for device deformationtogether with device properties; in response to an input by the user,applying a set of actuation controls which is based upon the propertiesof the distal end portion of the medical device that will configure thedistal end in the desired shape and orientation.
 52. A method ofspecifying a set of actuation controls to apply to a set of remotelyactuated elements at the distal end of a flexible medical device in anoperating region in a patient, the method comprising: identifying pointson the distal end of the medical device on an image of the operatingregion; based upon information about the distal end of the medicaldevice from the identified points, displaying a reconstructed virtualimage of the medical device that can be computationally manipulated asif a new magnetic field were applied to the device; accepting inputsfrom the user to change the configuration of the virtual image of themedical device.; applying actuation controls to actuate the actualmedical device in the operating region as computed from a combination ofuser-specified inputs and a computational model of device physics,corresponding to the user-manipulated virtual image.
 53. A method ofspecifying a set of actuation controls to apply to a set of remotelyactuated elements at the distal end of a flexible medical device in anoperating region in a patient, the method comprising: identifying pointson the distal end of the medical device on an image of the operatingregion; based upon information about the distal end of the medicaldevice from the identified points, displaying a reconstructed virtualimage of the medical device that can be computationally manipulated asif a new magnetic field were applied to the device; accepting inputsfrom the user identifying a target location on the image of theoperating region that the user desires the device tip to reach; applyingactuation controls to actuate the actual medical device in the operatingregion as computed from a combination of user-specified inputs and acomputational model of device physics, corresponding to theuser-specified tip location.
 54. A method of specifying a set ofactuation controls to apply to a set of remotely actuated elements atthe distal end of a flexible medical device in an operating region in apatient, the method comprising: use of device localization data at oneor more points on the device to reconstruct the current deviceconfiguration; displaying a reconstructed virtual image of the medicaldevice that can be computationally manipulated as if a new set ofactuation controls were applied to the device; accepting inputs from theuser to change the configuration of the virtual image of the medicaldevice; applying actuation controls to actuate the actual medical devicein the operating region as computed from a combination of user-specifiedinputs and a computational model of device physics, corresponding to theuser-manipulated virtual image.
 55. A method of specifying a set ofactuation controls to apply to a set of remotely actuated elements atthe distal end of a flexible medical device, the method comprising: useof device localization data at one or more points on the device toreconstruct the current device configuration; displaying a reconstructedvirtual image of the medical device that can be computationallymanipulated as if a new set of actuation controls were applied to thedevice; accepting inputs from the user to identify a target location onthe display that the user desires the device tip to reach; applyingactuation controls to actuate the actual medical device in the operatingregion as computed form a combination of user-specified inputs and acomputational model of device physics, corresponding to theuser-specified tip location.
 56. A method of navigating the distal endof a flexible medical device having a set of remotely actuated elementsthrough the application of a remote set of actuation controls, themethod comprising: identifying a plurality of points on two projectedimages of the distal end portion of the medical device while a known (orzero) set of actuation controls is applied to the distal end portion ofthe medical device; identifying a desired location for the tip of thedevice on one or both planes of projected images of the device and itsoperating environment, or on a three-dimensional regional electrical mapor its projections; computer calculation based upon the elastic andmagnetic properties of the device and automatic drawing of one or moreof: (a) a locus of possible locations of the medial device tip withinthe plane of the distal device tip, shown on one or both planes of theprojected images(s); (b) an accessible surface of possible locationsthat can be accessed by the device tip by a combination of deflectionand axial rotation, shown on one or both planes of the projectedimage(s); (c) said locus or accessible surface rendered together withdevice tip in a three-dimensional graphical view; identifying a desiredlocation for the tip of the device as a point on said locus or on saidaccessible surface in either image plane, three dimensional graphicalview, or regional electrical activity map; computation of a set ofactuation controls and device advancement/retraction that will enablethe device to closely reach the desired tip location; use of acomputer-controlled device advancer to control advancement andretraction of said device; instructing the computer to automaticallyadvance the device to the selected point by application of appropriateactuation controls and device advancer movements; and displaying one ormore of (a) a new set of projected images of the device and itsoperating environment, or (b) a new three-dimensional regionalelectrical map or its projections.
 57. A method of navigating the distalend of a flexible medical device having a set of remotely actuatedelements through the application of a set of actuation controls, themethod comprising: identifying a plurality of points on two projectedimages of the distal end portion of the medical device while a known (orzero) set of actuation controls is applied to the distal end portion ofthe medical device; identifying a desired location for the tip of thedevice on one or both planes of projected images of the device and itsoperating environment, or on a three-dimensional region electrical mapor its projections; computer calculation based upon the elastic andmagnetic properties of the device and automatic drawing of one or moreof: (a) a locus of possible locations of the medical device tip withinthe plan of the distal device tip, shown on one or both planes of thebi-plane image; (b) an accessible surface of possible locations that canbe accessed by the device tip by a combination of deflection and axialrotation, shown on one or both planes of the projected image(s); (c)said locus or accessible surface rendered together with device tip in athree-dimensional graphical view; identifying a desired location for thetip of the device as a point that is beyond said locus or saidaccessible surface in either image plane, three-dimensional graphicalview, or regional electrical activity map; computation of a set ofactuation controls and device advancement/retraction that will enablethe device to closely reach the desired tip location; use of acomputer-controlled device advancer to control advancement andretraction of said device; instructing the computer to automaticallyadvance the device to the selected point by application of appropriatemagnetic fields and device advancer movements; and displaying one ormore of (a) a new set of projected images of the device and itsoperating environment, or (b) a new three-dimensional regionalelectrical map or its projections.
 58. A method of navigating the distalend of a magnetically guideable catheter, whose distal tip is orientablewith an applied magnetic field, and whose free length from the distalend of a guide sheath is telescopingly adjustable, to a selected targetpoint in an operating region in a subject, the method comprising: usinga computational model of the catheter and information about the positionand orientation of the distal end of the guide sheath to determine themagnetic field to apply and to determine the free length to extend thecatheter from the guide sheath to cause the distlal tip to reach thetarget point.
 59. The method according to claim 58 further comprisingdisplaying a virtual image of the catheter as if the determined magneticfield was applied to the distal tip and the catheter was extended to thedetermined free length.
 60. The method according to claim 59 furthercomprising applying the determined magnetic field to the distal tip andextending the catheter from the guide sheath to the determined freelength.
 61. A method of navigating the distal end of an orientablecatheter, whose distal tip is orientable with an applied controlvariable, and whose free length from the distal end of a guide sheath istelescopingly adjustable, to a selected target point in an operatingregion in a subject, the method comprising: using a computational modelof the catheter and information about the position and orientation ofthe distal end of the guide sheath to determine the control variable toapply and to determine the free length to extend the catheter from theguide sheath to cause the distlal tip to reach the target point.